Skip to content

Add DeBERTa-ConPara-v2.14b#177

Open
MohamedMady19 wants to merge 2 commits into
liamdugan:mainfrom
MohamedMady19:submission/conpara-v214b
Open

Add DeBERTa-ConPara-v2.14b#177
MohamedMady19 wants to merge 2 commits into
liamdugan:mainfrom
MohamedMady19:submission/conpara-v214b

Conversation

@MohamedMady19

Copy link
Copy Markdown

Fixes preprocessing bug in v2.14 (PR #172).

Root cause: submission script missing unicode_normalize() before inference:

  • zero_width_space: collapsed to 2.2% TPR (expected ~97%+ with fix)
  • homoglyph: collapsed to 38.5% TPR (expected ~99%+ with fix)

All other 10 attack types v2.14 already outperformed v2.13 (98.6–99.4%).
Same checkpoint as v2.14, only inference preprocessing corrected.

@MohamedMady19

Copy link
Copy Markdown
Author

@liamdugan Please feel free to process this. Thank you!

@github-actions

Copy link
Copy Markdown

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

DeBERTa-ConPara-v2.14b

Release date: 2026-07-07

I've committed detailed results of this detector's performance on the test set to this PR.

Warning

Failed to find threshold values that achieve False Positive Rate(s): (['1%']) on all domains. This submission will not appear in the main leaderboard for those FPR values; it will only be visible within the splits in which the target FPR was achieved.
On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 95.70 and a TPR of 98.90% at FPR=5%.
Without adversarial attacks, it achieved AUROC of 96.18 and a TPR of 98.69% at FPR=5%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

@MohamedMady19

Copy link
Copy Markdown
Author

@liamdugan - these results look great. The TPR@1% warning is expected given the threshold calibration. Happy to proceed with merging as-is. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant